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Description 

SINGLE VISION SENSOR OBJECT 
DETECTION SYSTEM 

Background of Invention 

[0001] The present invention relates to veliicle sensing systems. 
More particularly, the present invention relates to a 
method and system for performing object detection utiliz- 
ing a single vision sensor. 

[0002] Various sensing systems currently exist for performing 

collision warning and countermeasure system operations, 
such as detection, classification, tracking, and relative 
distance and velocity estimation of objects within a close 
proximity of a host vehicle. Sensing systems also exist for 
performing other sensing system operations, such as oc- 
cupant related operations and adaptive cruise control op- 
erations. 

[0003] Collision warning and countermeasure system operations 
include providing a vehicle operator knowledge and 
awareness of vehicles and objects that are within a close 



proximity of tlie liost vehicle to prevent colliding with 
those objects. Countermeasure systems exist in various 
passive and active forms. Some countermeasure systems 
are used to aid in the prevention of a collision, others are 
used to aid in the prevention of injury to a vehicle opera- 
tor. 

[0004] Certain collision warning and countermeasure systems are 
able to sense an object within a close proximity of the 
host vehicle and warn the host vehicle operator such that 
the operator can take precautionary steps to prevent a 
collision or injury. Other collision warning and counter- 
measure systems activate passive or active countermea- 
sures such as airbags, load limiting seatbelts, or brake 
control whereby the system itself aids in the prevention of 
a collision or an injury. 

[0005] Occupant related operations include the detection of oc- 
cupant characteristics, the determination of which safety 
system countermeasures to perform, and the adjustment 
of deployment or enablement times and rates of the 
countermeasures. Example countermeasures that may be 
enabled are seat belt pretensioners and airbags. Occupant 
characteristics may include occupant positioning within a 
seat, occupant size, or other known occupant characteris- 



tics. 

[0006] Adaptive cruise control operations include adjusting a 

host vehicle traveling speed to maintain a safe operating 
distance between the host vehicle and a target vehicle. 
Other similar sensing systems also exist, such as lane de- 
parture and lane-keeping systems, which monitor lane 
markers or roadway lane designating lines and provide 
warnings when the host vehicle is not maintaining travel 
within a current operating lane. 

[0007] The various sensing systems may be ultrasonic, radar, li- 
dar, or vision-based. In order to perform each of the vari- 
ous sensing system tasks many of the stated sensors are 
distributed throughout the host vehicle. Not only does 
each of the sensors increase vehicle manufacturing and 
maintenance costs, but in order to perform object detec- 
tion and assessment multiple sensors are needed. 

[0008] Also, various techniques have been utilized, such as trian- 
gulation, in determining range, velocity, acceleration, and 
identification of objects in relation to a host vehicle. Typi- 
cally, two or more cameras are used to determine the 
stated parameters, potentially further increasing the num- 
ber of sensors used. 

[0009] One technique, that is used to reduce the number of sen- 



sors within a liost vehicle, uses vision sensors and per- 
forms optical flow visual image processing to estimate 
range of a target. In essence the technique infers three- 
dimensional characteristics from two-dimensional infor- 
mation. Other image processing techniques assume target 
size, based on a known range of vehicle sizes, and esti- 
mate range, based on the assumed size or an apparent 
two-dimensional size and image pixel intensity. The 
stated techniques tend to be inaccurate due to the as- 
sumptions and estimations contained therein. 
[0010] Thus, there exists a need for an improved sensing system 
that decreases the number of sensors to perform object 
detection and determines object parameters associated 
with the detected objects. Reduction in the number of 
sensors can aid in reducing the number of related vehicle 
components, system complexity, and vehicle manufactur- 
ing and maintenance costs. 
Summary of Invention 

[0011] The present invention provides a method and system for 
performing object detection and safety system operations 
within a vehicle. A sensing system for a vehicle is provided 
and includes a single vision sensor that has a position on 
the vehicle. The vision sensor detects an object and gen- 



erates an object detection signal. A controller is coupled 
to the vision sensor and generates a safety system signal 
in response to the position of the vision sensor and the 
object detection signal. 

[0012] The embodiments of the present invention provide several 
advantages. One of several of these advantages is the 
provision of a sensing system that is capable of determin- 
ing object parameters relative to a host vehicle through 
use of a single vision sensor. In so doing, the present in- 
vention minimizes the number of sensors and related sys- 
tem components needed to perform multiple safety sys- 
tem operations. The present invention is capable of per- 
forming the safety system operations that are traditionally 
performed through use of multiple vision sensors. 

[0013] Another advantage of the present invention is that it pro- 
vides a sensing system that utilizes a single vision sensor, 
but provides accurate measurements for improved sens- 
ing system performance. 

[0014] Furthermore, the present invention is capable of deter- 
mining object parameters without assuming actual size of 
a target to estimate or determine range of the target. 

[0015] The present invention itself, together with attendant ad- 
vantages, will be best understood by reference to the fol- 



lowing detailed description, taken in conjunction with the 

accompanying figures. 
Brief Description of Drawings 

[0016] For a more complete understanding of this invention ref- 
erence should now be made to embodiments illustrated in 
greater detail in the accompanying figures and described 
below by way of examples of the invention wherein: 

[0017] Figure 1 is a block diagrammatic view of a safety system 
for a vehicle in accordance with an embodiment of the 
present invention; 

[0018] Figure 2 is a logic flow diagram illustrating a method of 
performing safety system operations within a vehicle in 
accordance with an embodiment of the present invention; 

[0019] Figure 3A is a side view of the vehicle utilizing the safety 
system of Figure 1 and detecting an object in the same el- 
evation as the vehicle in accordance with an embodiment 
of the present invention; and 

[0020] Figure 3B is a side view of the vehicle utilizing the safety 
system of Figure 1 and detecting an object in a different 
elevation as the vehicle in accordance with an embodi- 
ment of the present invention. 
Detailed Description 



[0021] While the present invention is described with respect to a 
system and method of performing safety system opera- 
tions within a vehicle, the present invention may be 
adapted and applied to various systems including: colli- 
sion warning systems, collision avoidance systems, park- 
ing-aid systems, reversing-aid systems, passive counter- 
measure systems, adaptive cruise control systems, lane 
departure systems, lane-keeping systems, windshield 
clearing systems, or other systems known in the art. 

[0022] In the following description, various operating parameters 
and components are described for multiple constructed 
embodiments. These specific parameters and components 
are included as examples and are not meant to be limit- 
ing. 

[0023] Additionally, in the following description, the term "per- 
forming" may include activating, deploying, initiating, 
powering, and other terms known in the art that may de- 
scribe the manner in which a countermeasure or a comfort 
and convenience feature may be operated. 

[0024] As vvell, in the following description, various countermea- 
sures are discussed. The countermeasures may be re- 
versible or irreversible. Reversible countermeasures refer 
to countermeasures that may be reset to their original 



form or used repeatedly without a significant amount of 
functional deficiency, which may be determined by a sys- 
tem designer. Irreversible countermeasures refer to coun- 
termeasures such as airbags that, once deployed, are not 
reusable. 

[0025] Furthermore, a countermeasure signal may include infor- 
mation pertaining to the above-stated reversible and irre- 
versible countermeasures or may include other informa- 
tion, such as collision warning information, and parking- 
aid or reversing-aid countermeasure information. For ex- 
ample, the countermeasure signal may contain object de- 
tection information, which may be used to indicate to a 
vehicle operator the presence or close proximity of a de- 
tected object. 

[0026] In addition, the term "object" may refer to any animate or 
inanimate object. An object may be a vehicle, a pedes- 
trian, a lane marker, a road sign, a roadway lane designat- 
ing line, a vehicle occupant, or other object known in the 
art. 

[0027] Also, although the present invention is primarily described 
with respect to a single sensor, in applications where a 
single sensor is infeasible or undesirable additional sen- 
sors may be utilized and may be utilized in a similar fash- 



ion. 

[0028] Referring now to Figure 1, a blocl< diagrammatic view of a 
safety system 10 for a veliicle 12 in accordance with an 
embodiment of tlie present invention is sliown. Tlie safety 
system 10 may perform as a sensing system, an adaptive 
cruise control system, a countermeasure system, or other 
safety system known in the art. The safety system 10 in- 
cludes a single vision sensor 14 that has an associated 
position on the vehicle 12. The vision sensor 14 detects 
objects within proximity of the vehicle 12. The vision sen- 
sor 14 is coupled to a controller 16, which determines ob- 
ject related parameters in response to the position of and 
signals received from the vision sensor 14. The controller 
16 in response to the object parameters also performs 
various other safety system operations or tasks. 

[0029] The vision sensor 14 is coupled to a signal processor 18, 
which in turn is coupled to the controller 16. The signal 
processor 18 formats object detection signals generated 
by the vision sensor 14 for further processing by the con- 
troller 16. The vision sensor 14 may be a camera, a 
charged-coupled device, an infrared detector, a sensor 
formed of a series of photodiodes, a complementary 
metal-oxide semiconductor, or other vision sensor known 



in the art. In one embodiment of the present invention, 
the vision sensor 14 is a two-dimensional camera. The vi- 
sion sensor 14 may be located in various locations on the 
vehicle 12. 

[0030] The signal processor 18 receives and converts the object 
detection signals, generated by the vision sensor 14, into 
a proper format for reception by the controller 16. The 
signal processor 18 may include analog-to-digital con- 
verters, filters, or amplifiers, as well as other signal condi- 
tioning components known in the art. The signal proces- 
sor 18 may be part of the vision sensor 14, the controller 
16, may be a stand-alone device, as shown, or may be 
some combination thereof. 

[0031] The controller 16 may be microprocessor based such as a 
computer having a central processing unit, memory (RAM 
and/or ROM), and associated input and output buses. The 
controller 16 may be an application-specific integrated 
circuit or be formed of other logic devices known in the 
art. The controller 16 may be a portion of a central vehicle 
main control unit, an interactive vehicle dynamics module, 
a restraints control module, a main safety controller, or 
may be a stand-alone controller as shown. 

[0032] The controller 16 may perform various different sensing 



system operations including adaptive cruise control, lane- 
keeping control, lane-departure control, collision avoid- 
ance control, countermeasure control, or other sensing 
system operations known in the art. The operations may 
be performed sequentially or simultaneously. 

[0033] Adaptive cruise control is used for the monitoring of ob- 
jects forward of the vehicle 12 and for the maintaining of 
a safe predetermined distance away from the detected 
objects to prevent a collision therewith. When adaptive 
cruise control is active the controller 16 may warn a vehi- 
cle operator of an impending object or perform a counter- 
measure as to alter the speed of travel of the vehicle 12. 

[0034] Lane-keeping and lane-departure control refer to when 
the controller 16 monitors lane markings or roadway lane 
designating lines and warns the vehicle operator when the 
vehicle 12 is exiting a current lane of travel or is directed 
to exit the current lane of travel. The controller 16 may 
perform a countermeasure, such as one of the counter- 
measures 20, to maintain the current lane of travel. For 
example, the controller 16 may control vehicle steering to 
adjust the direction of travel of the vehicle 12. 

[0035] Countermeasure control may include occupant related op- 
erations, such as detecting occupant characteristics, de- 



termining which of the countermeasures to perform, and 
adjusting times and activating rates of the countermea- 
sures 20. The occupant characteristics may include occu- 
pant positioning within a seat, occupant size, or other 
l<nown occupant characteristics. 

[0036] The countermeasures 20 may include passive counter- 
measures 22 and active countermeasures 24. The con- 
troller 16 as part of a countermeasure may generate a 
warning signal and communicate the warning signal to 
vehicle occupants via the indicator 26. 

[0037] The passive countermeasures 22 may include internal air 
bag control, seatbelt control, knee bolster control, head 
restraint control, load limiting pedal control, load limiting 
steering control, pretensioner control, external air bag 
control, pedestrian protection control, and other passive 
countermeasures known in the art. 

[0038] The active countermeasures 24 may include brake control 
25 and steering control 27, as shown. The active counter- 
measures may also include throttle control, suspension 
control, transmission control, and other vehicle system 
controls known in the art. The controller 16 may signal 
the vehicle operator via the indicator 26 of an impending 
potential collision so that the vehicle operator may ac- 



lively perform a precautionary action, such as applying the 
brakes or steering to prevent a collision. 

[0039] The Indicator 26 is used to signal or indicate a safety sys- 
tem signal, which may include a warning signal, an adap- 
tive cruise control signal, a collision-warning signal, a 
countermeasure signal, or an object identification signal 
in response to the object detection signals. The indicator 
26 may include a video system, an audio system, an LED, 
a light, a global positioning system, a heads-up display, a 
headlight, a taillight, a display system, a telematic system, 
or other indicator known in the art. The indicator 26 may 
supply warning signals, collision-related information, lane 
departure and lane-keeping information, external-warn- 
ing signals to objects or pedestrians located outside of 
the vehicle 12, or other pre and post collision information. 

[0040] The controller 16 determines which of the sensing system 
operations to perform. The controller 16 while performing 
one or more of the sensing system operations may deter- 
mine whether to perform one or more of the countermea- 
sures 20 and indicate to the vehicle operator various ob- 
ject and vehicle status information. Depending upon the 
relative positions, velocities, and accelerations of the de- 
tected objects, the controller 16 may also determine 



whether to indicate to the vehicle operator of a potential 
collision or may perform a countermeasure, as needed, to 
prevent a collision, mitigate a potential injury, or prevent 
the vehicle 12 from traveling outside a current lane of 
travel. 

[0041] Various other vehicle and occupant sensors may be cou- 
pled to the controller 16 for determining whether to per- 
form the various safety system tasks. The sensors may in- 
clude a velocity sensor 28, a transmission gear sensor 30, 
occupant sensors 32, as shown, or other sensors known 
in the art. The velocity sensor 28 may be utilized to deter- 
mine relative range rate of an object. The transmission 
gear sensor 30 may be utilized to determine traveling di- 
rection of the vehicle 12. The occupant sensors 32 may be 
utilized to identify and determine occupant characteris- 
tics, such as size and position within the vehicle 12. 

[0042] A memory 34 may be coupled to the controller 16 for 
storing various vehicle, object, and occupant related in- 
formation. The memory 34 may be part of the vision sen- 
sor 14, the controller 16, may be a stand-alone device, as 
shown, or may be a combination thereof. The memory 34 
may, for example, store position information related to 
the vision sensor 14. The position information of the vi- 



sion sensor 14 is used by the controller 16 in determining 
object related parameters, which are described in further 
detail below. 

[0043] The safety system 10 may be located in various locations 
in the vehicle 12. The safety system 10 may be located 
within an overhead console, near any vehicle window, near 
a vehicle bumper, in or behind a vehicle dashboard, inter- 
nal to or external from a vehicle cabin, or in various other 
locations known in the art. 

[0044] Referring now to Figures 2, 3A, and 3B, a logic flow dia- 
gram illustrating a method of performing safety system 
operations within the vehicle 12 and side views of the ve- 
hicle 12 and a detected object 40 are shown in a first 
state and in a second state, respectively, according to the 
embodiments of the present invention. 

[0045] In step 100, the vision sensor 14 detects the object 40 

and generates an object detection signal. In step 102, the 
signal processor 18 converts, modifies, and formats the 
object detection signal for the reception by the controller 
16. The signal processor 18 may, for example, include an 
analog-to-digital converter to convert analog image sig- 
nals into a digital format. 

[0046] In step 104, the controller 16 determines the position and 



the orientation of tlie vision sensor 14 within and relative 
to a fixed reference point or set of reference points on the 
vehicle 12. In the embodiment of Figures 3A and 3B, a 
fixed reference point 42 is shown and represents a hood- 
line of the vehicle 12. The vision sensor 14, for the em- 
bodiment of Figures 3A and 3B, is mounted within an 
overhead console (not shown) of the vehicle 12 and posi- 
tion thereof is determined relative to the hoodline 42 of 
the vehicle 12. The position and orientation of the vision 
sensor 14 may be determined during the operation of the 
vehicle 12 or may be in the form of predetermined values 
that are stored in the memory and accessible by the con- 
troller 16. 

[0047] In step 106, the controller 16 determines the size and the 
up-angle of the object 40 in response to the position of 
the vision sensor 14 and the object detection signal. Up- 
angles a and a " correspond to the object 40 in states 1 
and 2, respectively. The up-angles a and a " have a first 
vector and a second vector, which are designated by 
and V in Figure 3A and V " and V " in Figures 3B. The 

2 3 12^ 

first vectors and " correspond to line segments 44 
that extend through the vision sensor 14 and the hoodline 
42. The second vectors V and V " correspond to line 

2 2 ^ 



segments 46 that extend through the vision sensor 14 
and a lower detected point 48 on the object 40. Of course, 
the up-angle may be determined using a point on the ob- 
ject 40. 

[0048] Notice in the embodiment of Figure 3A that the vehicle 12 
and the object 40 are at the same elevation or are on the 
same vertical plane and that the object 40 has a deter- 
mined apparent height H. The vertical or driving plane of 
a vehicle is typically horizontal. The following steps de- 
scribe change in the driving plane in a vertical direction. 
For example, a target in the same horizontal plane that 
moves away from a host vehicle may appear to rise verti- 
cally in an imaging view. Note that the driving planes of a 
target and of a host vehicle are not necessarily horizontal 
and may not be parallel. It is the transition from when the 
target and the host vehicle are on the same vertical plane 
to when they are on different vertical planes that are of 
interest. During the transition, change in image size re- 
veals whether the vertical rise of the target, within the 
imaging view, is due to an increase in range of the target 
or whether the target has merely moved onto a driving 
plane that is rising relative to the driving plane of the host 
vehicle. 



[0049] In Figure 3B, the vehicle 12 and the object 40 are in dif- 
ferent vertical planes and and the object 40 has an 
apparent height H" that is different from apparent height 
H. The apparent height of the object 40 is different be- 
tween state 1 and state 2 due to the object moving up a 
hill and altering the elevation or vertical plane by which it 
exists, thus, changing in apparent height. 

[0050] In step 108, the controller 16 in response to the deter- 
mined size and up-angle of the object 40 determines ob- 
ject related parameters, such as relative actual range and 
range rate of the object 40. The controller 16 is capable of 
determining the object related parameters from two- 
dimensional or three-dimensional information collected 
from the vision sensor 14. Height, width, and up-angle of 
the object 40 can be determined having just two- 
dimensional information from the vision sensor 14. The 
controller 16 initially or as an assumed default determines 
that the vehicle 12 and the object 40 are on the same ver- 
tical plane and monitors relative size and movement of 
the object 40 to determine whether this assumed default 
is true, and adjusts the operation accordingly. For exam- 
ple, after collecting and comparing a couple of frames of 
data the controller 16 is able to better determine vertical 



position of tlie object 40 relative to tlie veliicle 12. 
[0051] Distance between intersection points 50 for eacli vector 
pair V , V and V V ' are referred to as estimated 

^ 12 12 

ranges and R^ ' of tlie object 40. Tlie intersection points 

50 correspond witli tlie intersection of eacli line segment 

44, 44', 46, and 46' with the planes and P^. A relatively 

fixed horizontal distance D exists between the horizontal 

v 

position of the position sensor 14 and the intersection 
points 52 where the first vectors V and V intersect the 

^ 12 

plane P^. When the position sensor 14 is located in an up- 
per portion of the vehicle 12, such as in the overhead 
console, horizontal distance D^is approximately the same 
for various inclined roads, due to the high positioning of 
the vision sensor 14. Thus, horizontal distance D is ap- 
proximately the same for both state 1 and state 2 and is 
designated as such. 
[0052] Relative actual range R of the object 40, with respect to 
the vehicle 12, may be determined by adding the esti- 
mated range R^ with the fixed distance and subtracting 
a horizontal distance Vp between the vision sensor 14 and 
the hoodline 42. Relative actual ranges R and R ' for 

A A 

States 1 and 2 are represented by equations 1 and 2, re- 
spectively. 



[0053] R = R + D V (1) 
A E VP 

[0054] R ' = R ' + D V (2) 
A E VP 

[0055] As the object 40 becomes relatively closer or farther away 
from the vehicle 12 the apparent size of the object 40 in- 
creases or decreases in size accordingly. When using a 
two-dimensional sensor, size may refer to height and 
width of the object. The object 40 may appear to rise in 
elevation when the vision sensor 14 is situated in a higher 
vertical position than the object 40 and distance between 
the object 40 and the vehicle are increasing. When the 
size of the object 40 appears not to change between col- 
lected frames of information, but the object 40 appears to 
be moving upward or downward then the controller 16 
may determine that the vertical plane of the object 40 is 
changed. Actual range of the object 40 is altered accord- 
ing to the change in the vertical plane of the object 40. 
When the size of the object 40 does appear to change 
than the controller 16 may determine that the actual 
range has also changed. 

[0056] The situation also exists where the vehicle 12 and the ob- 
ject 40 exist on different planes, such that the vertical rel- 
ative position of the object 40 is unaltered, but yet the 



distance between the vehicle 12 and the object 40 is 
changed. In this situation, the controller 16 monitors and 
determines that the size of the object 40 is changed and 
in response thereto determines change in the actual 
range. 

[0057] In step 110, the controller 16 generates safety system 

signals in response to the apparent size and relative ele- 
vation of the object 40. The controller 16 may also gener- 
ate the safety system signals in response to other known 
object related parameters. 

[0058] In step 112, the controller 16 may determine occupant 
characteristics using the occupant sensors 30. Step 112 
may be performed simultaneously with step 110, whereby, 
the controller 16 generates or modifies the safety system 
signals in response to the determined occupant character- 
istics. 

[0059] In step 114, the controller 16 may perform a countermea- 
sure 20 in response to the safety system signals. Perfor- 
mance of the countermeasure 20 safety system signal may 
be in response to any of the object related parameters, 
the occupant characteristics, other parameters known in 
the art, alone or in a combination thereof. The controller 
16 may perform a countermeasure operation, an adaptive 



cruise control operation, or any of the other above- 
mentioned or l<nown in the art safety system operations 
or tasks in response to the safety system signal. In one 
embodiment of the present invention, the controller 16, 
as part of an adaptive cruise control function, reduces the 
speed of the vehicle 12 to avoid colliding with the object 
40. The controller 16 returns to step 100 or 104 to detect 
additional objects and further monitor any detected ob- 
jects that are of concern. 
[0060] The above-described steps are described with respect to 
the vertical and horizontal positioning of an object relative 
to a vehicle; they may be easily modified to also perform 
operation with respect to lateral positioning of the object 
relative to the vehicle. Also, although the above-described 
steps are described with respect to determining up-angle 
of a detected object, other angles may be utilized in de- 
termining object related parameters. The above-described 
steps are also meant to be illustrative examples; the steps 
may be performed sequentially, synchronously, simulta- 
neously, or in a different order depending upon the appli- 
cation. 

[0061] The present invention provides an accurate safety system 
for a vehicle that is capable of determining object param- 



eters, such as object range and range rate, utilizing a sin- 
gle vision sensor. The present invention in utilizing a sin- 
gle vision sensor minimizes the number of sensors and 
sensing system components within a vehicle, minimizes 
sensing system complexity, and minimizes manufacturing 
and maintenance costs involved therein. 
[0062] While the invention has been described in connection with 
one or more embodiments, it is to be understood that the 
specific mechanisms and techniques which have been de- 
scribed are merely illustrative of the principles of the in- 
vention, numerous modifications may be made to the 
methods and apparatus described without departing from 
the spirit and scope of the invention as defined by the ap- 
pended claims. 



